Many companies are developing autonomous vehicles for commercial and personal use on existing roadways for a variety of applications, including but not limited to personal taxi services, delivery services, and the like. In accordance with the present invention, an autonomous vehicle is a vehicle capable of operating without a human driver. Such vehicles can be designed to operate utilizing an onboard computer and a system of sensors designed to drive, steer, brake, and otherwise operate the vehicle in the same manner as if there were a human operator. It is envisioned that fleets of autonomous vehicles will soon be available, similar to a network of taxis, buses or delivery vans, whereby a user can request an autonomous vehicle to pick-up, transport and drop off passengers, or pick-up, transport, and deliver packages or the like, on demand. Alternatively, users can own an autonomous vehicle for personal use and use it for ordinary tasks, such as commuting to work, running errands, dropping the kids off at school, for travel, or the like.
Current autonomous vehicles in the development and testing stages generally utilize multiple systems to fully operate the vehicle without a human operator. First, a standard GPS system is used to plan a route for the vehicle. Taking into account the starting point and the destination for a trip, as well as other factors such as traffic conditions, road closures, weather conditions, preferred routes, toll roads, etc., the GPS system determines the best route for the vehicle to take. However, for safe and efficient operation, autonomous vehicles also need a system to recognize dynamic conditions along the route during operation of the vehicle. Such a system may be referred to as an augmented GPS system, which utilizes an array of technologies, such as cameras, sensors, radar, LIDAR and lasers to provide a three-dimensional view around the vehicle during operation. Such a system can generally keep track of other cars around the vehicle; detect obstacles or hazards around the car, in the road up ahead, or approaching the car from the front, rear, or sides; and determine the location of the edge of the road or travel lane, upcoming turns, hills or descents, and assess general road conditions ahead, behind and around the vehicle.
Autonomous vehicles also require a centralized system within the car to process the information provided from the GPS system and augmented GPS system and utilize the processed information to operate the vehicle. Such commonly utilized systems generally include a Computer Area Network (CAN) bus in the vehicle to communicate with and coordinate operation of the GPS system, augmented GPS system and other autonomous vehicle operating systems.
Non-autonomous vehicles also utilize similar technology to back-up a human driver. For example, cars have used various forms of cruise control for decades. More recently, cars have been equipped with systems that will autonomously parallel park the car. Many modern cars are now equipped with systems that assist the driver when the car begins to drift out of its lane on the highway, or brake the car if it is getting too close to the car in front of it, or alert the driver if there is an object in the road ahead.
Until guidance systems on-board autonomous vehicles match or exceed the perception and analytical decision-making ability of human drivers, there will be numerous ongoing daily situations which will frustrate the ability of a full autonomous vehicle to properly and dynamically respond to, or react to, its surroundings. Moreover, until autonomous vehicles can safely rely upon existing operational systems and sensors for safe and efficient operation and eliminate essentially all risks, the public will continue to be hesitant to put full faith in true autonomous operation of such vehicles. Indeed, “real-world” autonomous vehicular tests have shown to have had numerous guidance failures, accidents, etc., caused by vehicles, pedestrians, or bikers laterally crossing into a travel lane that existing guidance systems and sensors have either failed to adequately detect, recognize and/or react to these situations in a timely fashion, or have had a high rate of “false” indications. Even with a plethora of guidance systems and sensors in both autonomous and driven-vehicles, there have been failures to identify and avoid interaction with pedestrian and/or biker traffic for various reasons.
While various optically-based automotive and/or autonomous guidance systems and sensors (e.g., video, LIDAR, etc.) are capable of performing well under ideal visual conditions, their capabilities can diminish to unusable levels under adverse ambient conditions, such as during or shortly after rain, snowfall, fog, etc., or when it is dark outside and in low-lighted areas of the roadway, or even when the pedestrian blends into the surroundings based upon clothing, skin tone, weather, sun glare, etc. Additionally, while the existing level of “on-board” sensors, cameras, devices, and interfaces can alter an autonomous vehicle's driving characteristics to a limited degree (e.g., by braking for unexpected obstacles and/or other vehicles, or steering a vehicle if it drifts out of its lane), there is currently an inherent extreme deficiency with specifically assisting autonomous vehicles in the identification of pedestrian traffic and furthermore determining when said pedestrian traffic is poised to negatively interact with a vehicle, and further assist such vehicles in automatically taking action to prevent such negative interactions. Accordingly, there is a need to improve upon existing optically-based guidance systems and sensors to ensure that operation of an autonomous vehicle is safe and efficient in all conditions.
It has further proven most daunting to operate a vehicle autonomously under any and all conditions due to the inherent difficulties of requiring a vehicle to make real-time decisions in view of unusual, unexpected or unpredicted situations. Even under ideal ambient conditions, the CAN bus of the vehicle must coordinate a considerable number of sensors, and requires extensive analytical processing power to merely allow the vehicle to try to identify what expected and/or unusual or unexpected objects are during operation, with very little time to actually detect, recognize and react appropriately. Since a primary object of the present invention is to first detect a “human”, traditional optical means typically fail as there are an infinite number of visible representations of a “human” that could not all possibly reside in an on-board database. An example of this would be a video processor trying to identify a “human” amongst so many variations such as: a human propelling themselves in a wheelchair versus a human in a wheelchair being pushed by another human versus a human viewed in profile standing next to a bicycle versus a human in profile wearing a backpack, etc. Further, even if the vehicle is able to identify an object or situation, there is still a challenge with having the vehicle then attempt to figure out what the correct procedural response should be. As there are almost an infinite number of potential scenarios which require an appropriate response by the autonomous vehicle, this again proves to be exceedingly difficult, if not impossible, given the limitations of having a constrained amount of on-board processing power and database size in the vehicle, as well as little real time available for a response. Given that there are so many unknown or unpredicted situations, a better approach to autonomous vehicle guidance is still needed, and the best place to start is improving the speed, efficiency, and efficacy with which a vehicle is able to detect and identify an unknown, unexpected and unpredicted situation and a need for guidance adjustment.
Accordingly, there is a need for an improved system for the operation of an autonomous vehicle for identifying unknown, unexpected and unpredicted situations or conditions on a roadway that overcomes the drawbacks and limitations of existing dynamic guidance systems. Further, there is a need for a system that utilizes infra-red imaging sensors designed for specific “human” detection and analysis that can assist in the safe and efficient operation of vehicles in response to unknown, unexpected and unpredicted situations involving humans, whether they be pedestrians, runners, bikers, or any situation which causes them to impinge onto a roadway and into the predicted pathway of moving vehicles in conflict with vehicular traffic.